Color image input devices, such as a digital still camera and a video camera, generally have a configuration in which an optical filter of three colors of red (R), green (G), and blue (B) is incorporated in an image sensor. Light incident on such a color image input device is decomposed by the three-color optical filter and converted into signals corresponding to R, G, and B colors, respectively, by the image sensor.
When a silicon-based sensor is used as the image sensor used for the color image input device, sensitivity of the sensor ranges from a visible range to a near-infrared range. However, a near-infrared light component may have an adverse effect on color reproduction. However, the three-color optical filter guarantees a constant transmittance for a wavelength range corresponding to each color, but does not necessarily guarantee an optical transmission characteristic for ranges other than the visible range, such as the near-infrared range.
FIG. 17 is a diagram exemplifying a spectral transmittance of an RGB three-color optical filter. For example, when the visible range is set from 400 to 700 nm, it is expected that the filters for the respective colors have characteristics for transmitting light having a wavelength in the vicinity of 400 to 500 nm (B), 500 to 600 nm (G), and 600 to 700 nm (R). However, the filters for the respective colors may also have characteristics for transmitting light in a range other than the visible range as illustrated in FIG. 17.
Spectral sensitivity characteristics of an image sensor using a photodiode that is often adopted in an image input device include sensitivity in a wavelength range of 700 nm or more. In this case, when only applying a three-color optical filter having spectral sensitivity characteristics as illustrated in FIG. 17 to a typical image sensor, this may cause a problem in terms of color reproducibility. Accordingly, when the image sensor is required to have high color reproducibility, the image sensor is provided with an infrared cut filter.
FIG. 18 is a diagram illustrating color-matching functions of an XYZ colorimetric system associated with human color perception. As illustrated in FIG. 18, the human color perception does not include sensitivity for light in a wavelength of 700 nm or more. Accordingly, light having power in a wavelength range of 700 nm or more does not affect a perceived color that is a psychophysical value.
Herein, a case is assumed where light having power in a wavelength range of 600 nm or more as illustrated in FIG. 19 is observed. This light is perceived as red by a human. On the other hand, when the light is observed by an image sensor using a three-color optical filter as illustrated in FIG. 17, the output signal includes not only an R value, but also G and B values. Accordingly, the output signal represents a color different from the color (red) perceived by a human.
In the color image input device, an infrared cut filter having a spectral transmittance for removing an effect of near-infrared light in a wavelength of 700 nm or more as illustrated in FIG. 20 is used in order to implement color reproducibility according to the human color perception. Specifically, as illustrated in FIG. 21, the optical system of the color image input device is provided with an infrared cut filter 610 to thereby block incidence of near-infrared light on a three-color optical filter 620 and an image sensor 630. This configuration allows light having no power in the near-infrared range to be incident on the three-color optical filter 620 and the image sensor 630.
On the other hand, in a case of capturing a video under a circumstance in which an amount of light is insufficient, high-sensitivity capturing in which noise is suppressed is required. In such a case, it is desirable to increase an amount of received light in the image sensor in order that sensor noise due to an insufficient amount of light is suppressed. As a method for implementing high-sensitivity capturing in a dark place, a capturing method using near-infrared light is known.
A simplest method using near-infrared light during high-sensitivity capturing is a method in which an infrared cut filter set in an optical system is mechanically moved during high-sensitivity capturing to thereby temporarily remove the infrared cut filter from the optical system. However, this method has problems of an increase in the number of components, i.e., an increase in cost, as well as an increase in possibility of occurrence of a failure due to requirement for a mechanical operation for moving the infrared cut filter.
On the other hand, NPL 1 discloses a method for capturing without requiring any mechanical operation. Specifically, NPL 1 describes a capturing method using two cameras for capturing a color image and a near-infrared image, respectively.
Further, NPL 2 discloses, as illustrated in FIG. 22, an image sensor 700 having a configuration in which a four-color optical filter obtained by adding an infrared (IR) filter for transmitting near-infrared light to an RGB three-color optical filter is incorporated. FIG. 2 of NPL 2 illustrates spectral sensitivity characteristics of respective optical filters for R, G, B, and IR. The spectral sensitivity characteristics of respective optical filters for R, G, and B include spectral sensitivity similar to that of the IR filter in a near-infrared range. In order to implement high color reproducibility during capturing in the daytime, it is necessary to suppress or eliminate an effect of near-infrared light included in R, G, and B color signals. The image sensor described in NPL 2 removes IR components included in the R, G, and B color signals during capturing in the daytime, and uses not only an IR signal obtained by causing light to transmit an IR filter, but also IR components included in the R, G, and B color signals during capturing at night, thereby obtaining a black-and-white image.
PTL 1 discloses an imaging device that generates signals of R, G, B, and near-infrared (NIR) colors by using an R, G, and B three-color optical filter for transmitting NIR light, and a photosensor for detecting near-infrared light. This photo sensor includes a visible light sensor unit at a shallow position in a light incident direction, and also includes a non-visible light sensor unit at a deep position in the direction.
In addition, NPL 3 discloses a method for generating a four-channel image by separating a color channel and an NIR channel from an image captured by using a color filter array different from a typical one, by using two types of filters having different spectral transmission characteristics for G filters of an RGB Bayer type color filter array (CFA), without using an IR cut filter, and the like.